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Abstract 

Most natural language processing tasks re- 
quire lexical semantic information. Au- 
tomated acquisition of this information 
would thus increase the robustness and 
portability of NLP systems. This pa- 
per describes an acquisition method which 
makes use of fixed correspondences be- 
tween derivational affixes and lexical se- 
mantic information. One advantage of this 
method, and of other methods that rely 
only on surface characteristics of language, 
is that the necessary input is currently 
available. 

1 Introduction 

Some natural language processing (NLP) tasks can 
be performed with only coarse-grained semantic in- 
formation about individual words. For example, 
a system could utilize word frequency and a word 
cooccurrence matrix in order to perform informa- 
tion retrieval. However, many NLP tasks require at 
least a partial understanding of every sentence or 
utterance in the input and thus have a much greater 
need for lexical semantics. Natural language gen- 
eration, providing a natural language front end to 
a database, information extraction, machine trans- 
lation, and task-oriented dialogue understanding all 
require lexical semantics. The lexical semantic in- 
formation commonly utilized includes verbal argu- 
ment structure and selectional restrictions, corre- 
sponding nominal semantic class, verbal aspectual 
class, synonym and antonym relationships between 
words, and various verbal semantic features such as 
causation and manner. 

Machine readable dictionaries do not include 
much of this information and it is difficult and time 
consuming to encode it by hand. As a consequence, 
current NLP systems have only small lexicons and 



thus can only operate in restricted domains. Auto- 
mated methods for acquiring lexical semantics could 
increase both the robustness and the portability of 
such systems. In addition, such methods might pro- 
vide insight into human language acquisition. 

After considering different possible approaches to 
acquiring lexical semantic information, this paper 
concludes that a "surface cueing" approach is cur- 
rently the most promising. It then introduces mor- 
phological cueing, a type of surface cueing, and dis- 
cusses an implementation. It concludes by evalu- 
ating morphological cues with respect to a list of 
desiderata for good surface cues. 

2 Approaches to Acquiring Lexical 
Semantics 

One intuitively appealing idea is that humans ac- 
quire the meanings of words by relating them to 
semantic representations resulting from perceptual 
or cognitive processing. For example, in a situation 
where the father says Kim is throwing the ball and 
points at Kim who is throwing the ball, a child might 
be able learn what throw and ball mean. In the 
human language acquisition literature, Grimshaw 
(1981) and Pinker (1989) advocate this approach; 
others have described partial computer implementa- 
tions: Pustejovsky (1988) and Siskind (1990). How- 
ever, this approach cannot yet provide for the auto- 
matic acquisition of lexical semantics for use in NLP 
systems, because the input required must be hand 
coded: no current artificial intelligence system has 
the perceptual and cognitive capabilities required to 
produce the needed semantic representations. 

Another approach would be to use the semantics 
of surrounding words in an utterance to constrain 
the meaning of an unknown word. Borrowing an 
example from Pinker (1994), upon hearing I glipped 
the paper to shreds, one could guess that the mean- 
ing of glib has something to do with tearing. Sim- 
ilarly, one could guess that flip means something 



like eat upon hearing I filped the delicious sandwich 
and now I'm full. These guesses are cued by the 
meanings of paper, shreds, sandwich, delicious, full, 
and the partial syntactic analysis of the utterances 
that contain them. Granger (1977), Berwick (1983), 
and Hastings (1994) describe computational systems 
that implement this approach. However, this ap- 
proach is hindered by the need for a large amount 
of initial lexical semantic information and the need 
for a robust natural language understanding system 
that produces semantic representations as output, 
since producing this output requires precisely the 
lexical semantic information the system is trying to 
acquire. 

A third approach does not require any semantic 
information related to perceptual input or the in- 
put utterance. Instead it makes use of fixed cor- 
respondences between surface characteristics of lan- 
guage input and lexical semantic information: sur- 
face characteristics serve as cues for lexical seman- 
tics of the words. For example, if a verb is seen 
with a noun phrase subject and a sentential comple- 
ment, it often has verbal semantics involving spa- 
tial perception a nd cognition, e.g., believe, think 
worry, and see ( Fisher, Gleitman, and Gleitman 



1991; Gleitman, 199C). Similarly, the occurrence 



of a verb in the progressive tense can be used as 
a cue for the non-stativeness of the verb (Dorr 
and Lee, 1992| ); stative verbs cannot appear in the 
progress tense {e.g.* Mary is loving her new shoes). 
Another example is the use of patterns such as 
NP, NP * ,and otherNP to find lexical se mantic 
information such as hyponym (Hearst, 1992). Tem- 
ples, treasuries, and other important civic buildings 
is an example of this pattern and from it the infor- 
mation that temples and treasuries are types of civic 
buildings would be cued. Finally, inducing lexical 
semantics from distributional data {e.g., (Brown et 
al., 1992; Church et al., 1989| )) is also a form of sur- 
face cueing. It should be noted that the set of fixed 
correspondences between surface characteristics and 
lexical semantic information, at this point, have to 
be acquired through the analysis of the researcher — 
the issue of how the fixed correspondences can be 
automatically acquired will not be addressed here. 

The main advantage of the surface cueing ap- 
proach is that the input required is currently avail- 
able: there is an ever increasing supply of on- 
line text, which can be automatically part-of-speech 
tagged, assigned shallow syntactic structure by ro- 
bust partial parsing systems, and morphologically 
analyzed, all without any prior lexical semantics. 

A possible disadvantage of surface cueing is that 
surface cues for a particular piece of lexical semantics 



might be difficult to uncover or they might not exist 
at all. In addition, the cues might not be present 
for the words of interest. Thus, it is an empirical 
question whether easily identifiable abundant sur- 
face cues exist for the needed lexical semantic infor- 
mation. The next section explores the possibility of 
using derivational affixes as surface cues for lexical 
semantics. 

3 Morphological Cues for Lexical 
Semantic Information 

Many derivational affixes only apply to bases with 
certain semantic characteristics and only produce 
derived forms with certain semantic characteristics. 
For example, the verbal prefix un- applies to telic 
verbs and produces telic derived forms. Thus, it is 
possible to use un- as a cue for telicity. By search- 
ing a sufficiently large corpus we should be able to 
identify a number of telic verbs. Examples from the 
Brown corpus include clasp, coil, fasten, lace, and 
screw. 

A more implementation-oriented description of 
the process is the following: (i) analyze affixes by 
hand to gain fixed correspondences between affix and 
lexical semantic information (ii) collect a large cor- 
pus of text, (iii) tag it with part-of-speech tags, (iv) 
morphologically analyze its words, (v) assign word 
senses to the base and the derived forms of these 
analyses, and (vi) use this morphological structure 
plus fixed correspondences to assign semantics to 
both the base senses and the derived form senses. 
Step (i) amounts to doing a semantic analysis of a 
number of affixes the goal of which is to find se- 
mantic generalizations for an affix that hold for a 
large percentage of its instances. Finding the right 
generalizations and stating them explicitly can be 
time consuming but is only performed once. Tagging 
the corpus is necessary to make word sense disam- 
biguation and morphological analysis easier. Word 
sense disambiguation is necessary because one needs 
to know which sense of the base is involved in a 
particular derived form, more specifically, to which 
sense should one assign the feature cued by the affix. 
For example, stress can be either a noun the stress 
on the third syllable or a verb the advisor stressed 
the importance of finishing quickly. Since the suffix 
-ful applies to nominal bases, only a noun reading is 
possible as the stem of stressful and thus one would 
attach the lexical semantics cued by -ful to the noun 
sense. However, stress has multiple readings even 
as a noun: it also has the reading exemplified by 
the new parent was under a lot of stress. Only this 
reading is possible for stressful. 



In order to produce the results presented in the 
next section, the above steps were performed as fol- 
lows. A set of 18 affixes were analyzed by hand pro- 
viding the fixed correspondences between cue and 
semantics. The cued lexical semantic information 



was axiomatized using Episodic Logic (Hwang and 
Schubert, 1993| ) , a situation-based extension of stan- 
dard first order logic. The Penn Treebank ver- 



sion of the Brown corpus (Marcus, Santorini, and 
Marcinkiewicz, 1993) served as the corpus. Only 
its words and part-of-speech tags were utilized. Al- 
though these tags were corrected by hand, part-of- 
speech tagging can be automat ically performed with 



an er ror rate of 3 to 4 percent ( Mcrialdo, 1994 ; Brill 



1994|). Th e Alvey morphological analyzer ( [Ritchie et 
al., 1992j ) was used to assign morphological struc- 
ture. It uses a lexicon with just over 62,000 en- 
tries. This lexicon was derived from a machine read- 
able dictionary but contains no semantic informa- 
tion. Word sense disambiguation for the bases and 
derived forms that could not be resolved using part- 
of-speech tags was not performed. However, there 
exist systems for such word sense disambiguation 
which do not require explicit lexical semantic infor- 
mation (Yarowsky, 1993; [Bchutzc, 1992 ). 

Let us consider an example. One sense of the suf- 
fix -ize applies to adjectival bases (e.g., centralize). 
This sense of the affix will be referred to as -Aize. 
(A related but different sense applies to nouns, e.g., 
glamorize. The part-of-speech of the base is used 
to disambiguate these two senses of -ize.) First, 
the regular expressions ".*IZ(E|ING|ES|ED)$" and 
" A V.*" are used to collect tokens from the corpus 
that were likely to have been derived using -ize. The 
Alvey morphological analyzer is then applied to each 
type. It strips off -Aize from a word if it can find 
an entry with a reference form of the appropriate or- 
thographic shape and has the features "uninflected," 
"latinate," and "adjective." It may also build an ap- 
propriate base using other affixes, e.g., ^tradition -at\ 
-Aize]^ Finally, all derived forms are assigned the 
lexical semantic feature change-of-state and all 
the bases are assigned the lexical semantic feature 
IZE-DEPENDENT. Only the change-of-state fea- 
ture will be discussed here. It is defined by the axiom 
below. 



1 In an alternative version of the method, the mor- 
phological analyzer is also able to construct a base on 
its own when it is unable to find an appropriate base 
in its lexicon. However, these "new" bases seldom cor- 
respond to actual words and thus the results presented 
here were derived using a morphological analyzer config- 
ured to only use bases that are directly in its lexicon or 
can be constructed from words in its lexicon. 



For all predicates P with features 
CHANGE-OF-STATE and DYADIC: 
Vx,y,e [P(x,y)**e -> 

[3el: [at-end-of (el.e) A 
cause(e,el)] 
[rstate(P) (y)**el] A 
3e2 : at-beginning-of (e2 , e) 
[-irstate(P) (y)**e2]]] 

The operator ** is analogous to |= in situation 
semantics; it indicates, among other things, that a 
formula describes an event. P is a place holder for 
the semantic predicate corresponding to the word 
sense which has the feature. It is assumed that each 
word sense corresponds to a single semantic predi- 
cate. The axiom states that if a change-of-state 
predicate describes an event, then the result state of 
this predicate holds at the end of this event and that 
it did not hold at the beginning, e.g., if one wants to 
formalize something it must be non-formal to begin 
with and will be formal after. The result state of an 
-Aize predicate is the predicate corresponding to its 
base; this is stated in another axiom. 

Precision figures for the method were collected as 
follows. The method returns a set of normalized 
(i.e., uninflected) word/feature pairs. A human then 
determines which pairs are "correct" where correct 
means that the axiom defining the feature holds for 
the instances (tokens) of the word (type) . Because of 
the lack of word senses, the semantics assigned to a 
particular word is only considered correct, if it holds 
for all senses occurring in the relevant derived word 
tokens.^] For example, the axiom above must hold 
for all senses of centralize occurring in the corpus 
in order for the centralize/ change-of-state pair 
to be correct. The axiom for ize-DEPENDENT must 
hold only for those senses of central that occur in the 
tokens of centralize for the central / IZE-DEPENDENT 
pair to be correct. This definition of correct was con- 
structed, in part, to make relatively quick human 
judgements possible. It should also be noted that 
the semantic judgements require that the seman- 
tics be expressed in a precise way. This discipline 
is enforced in part by requiring that the features be 
axiomatized in a denotational logic. Another argu- 
ment for such an axiomatization is that many NLP 
systems utilize a denotational logic for representing 
semantic information and thus the axioms provide a 
straightforward interface to the lexicon. 

To return to our example, as shown in Table 1, 
there were 63 -Aize derived words (types) of which 



Although this definition is required for many cases, 
in the vast majority of the cases, the derived form and 
its base have only one possible sense (e.g., stressful). 



78 percent conform to the CHANGE-OF-STATE ax- 
iom. Of the bases, 80 percent conform to the ize- 
dependent axiom which will be discussed in the 
next section. Among the conforming words were 
equalize, stabilize, and federalize. Two words that 
seem to be derived using the -ize suffix but do not 
conform to the CHANGE-OF-STATE axiom are "penal- 
ize and socialize (with the guests). A different sort 
of non-conformity is produced when the morpholog- 
ical analyzer finds a spurious parse. For example, it 
analyzed subsidize as [sub- [side -ize]] and thus pro- 
duced the sidize/ CHANGE-OF-STATE pair which for 
the relevant tokens was incorrect. In the first sort, 
the non-conformity arises because the cue does not 
always correspond to the relevant lexical semantic 
information. In the second sort, the non-conformity 
arises because a cue has been found where one does 
not exist. A system that utilizes a lexicon so con- 
structed is interested primarily in the overall preci- 
sion of the information contained within and thus 
the results presented in the next section conflate 
these two types of false positives. 

4 Results 

This section starts by discussing the semantics of 18 
derivational affixes: re-, un-, de-, -ize, -en, -ify, -le, 
-ate, -ee, -er, -ant, -age, -ment, mis-, -able, -ful, - 
less, and -ness. Following this discussion, a table of 
precision statistics for the performance of these sur- 
face cues is presented. Due to space limitations, the 
lexical semantics cued by these affixes can only be 
loosely specified. However, they have been axiom- 
atized in a fashion exemplified by the CHANGE-OF- 



STATE axiom above (see (Light, 1996; Light, 1992)). 

The verbal prefixes un-, de-, and re- cue aspec- 
tual information for their base and derived forms. 
Some examples from the Brown corpus are unfas- 
ten, unwind, decompose, defocus, reactivate, and 
readapt. Above it was noted that un- is a cue for 
felicity In fact, both un- and de- cue the CHANGE- 
OF-STATE feature for their base and derived forms — 
the change-of-state feature entails the telic fea- 
ture. In addition, for un- and de-, the result state of 
the derived form is the negation of the result state of 
the base (neg-of-base-is-rstate), e.g., the result 
of unfastening something is the opposite of the result 
of fastening it. As shown by examples like reswim 
the last lap, re- only cues the telic feature for its 
base and derived forms: the lap might have been 
swum previously and thus the negation of the result 



the same as activating it. In fact, if one reactivates 
something then it is also being activated: the derived 
form entails the base (entails-base). Finally, for 
re-, the derived form entails that its result state held 
previously, e.g., if one recentralizes something then 
it must have been central at some point previous to 
the event of recentralization (PRESUPS-RSTATE) . 

The suffixes -Aize, -Nize, -en, -Aify, -Nify all 
cue the change-of-state feature for their derived 
form as was discussed for -Aize above. Some ex- 
emplars are centralize, formalize, categorize, colo- 
nize, brighten, stiffen, falsify, intensify, mummify, 
and glorify. For -Aize, -en and -Aify a bit more can 
be said about the result state: it is the base predi- 
cate (rstate-EQ-BASe), e.g., the result of formaliz- 
ing something is that it is formal. Finally -Aize, -en, 
and -Aify cue the following feature for their bases: 
if a state holds of some individual then either an 
event described by the derived form predicate oc- 
curred previously or the predicate was always true 
of the individual (ize-dependent), e.g., if some- 
thing is central then either it was centralized or it 
was always central. 

The "suffixes" -le and -ate should really be called 
verbal endings since they are not suffixes in English, 
i.e., if one strips them off one is seldom left with a 
word. (Consequently, only regular expressions were 
used to collect types; the morphological analyzer was 
not used.) Nonetheless, they cue lexical semantics 
and are easily identified. Some examples are chuckle, 
dangle, alleviate, and assimilate. The ending -ate 
cues a change-of-state verb and -le an activity 
verb. 

The derived forms produced by -ee, -er, and -ant 
all refer to participants of an event described by their 
base (part-in-e). Some examples are appointee, de- 
portee, blower, campaigner, assailant, and claimant. 
In addition, the derived form of -ee is also sentient 
of this event and non-volitional with respect to it 



state does not have to have held previously ( Dowty. 
1979). For re-, the result state of the derived form 



is the same as that of the base (rstate-eq-base- 
rstate), e.g., the result of reactivating something is 



( [Barker, 1995| ). 

The nominalizing suffixes -age and -ment both 
produce derived forms that refer to something re- 
sulting from an event of the verbal base predicate. 
Some examples are blockage, seepage, marriage, pay- 
ment, restatement, shipment, and treatment. The 
derived forms of -age entail that an event occurred 
and refer to something resulting from it (event- 
and-resultant)), e.g., seepage entails that seep- 
ing took place and that the seepage resulted from 
this seeping. Similarly, the derived forms of -ment 
entail that an event took place and refer either to 
this event, the proposition that the event occurred, 
or something resulting from the event (refers-to- 
e-OR-prop-OR-RESULt), e.q., a restatement entails 



that a restating occurred and refers either to this 
event, the proposition that the event occurred, or to 
the actual utterance or written document resulting 
from the restating event. (This analysis is based on 
JZucchi, 19891 ).) 

The verbal prefix mis-, e.g., miscalculate and mis- 
quote, cues the feature that an action is performed 
in an incorrect manner (incorrect-manner). The 
suffix -able cues a feature that it is possible to per- 
form some action (able-to-be-performed), e.g., 
something is enforceable if it is possible that some- 
thing can enforce it (Dowty, 1979). The words de- 
rived using -ness refer to a state of something having 
the property of the base (state-of-having-prop- 
OF-base), e.g., in Kim's fierceness at the meeting 
yesterday was unusual the word fierceness refers to 
a state of Kim being fierce. The suffix -ful marks 
its base as abstract (abstract): careful, peaceful, 
powerful, etc. In addition, it marks its derived form 
as the antonym of a form derived by -less if it exists 
(less- antonym). The suffix -less marks its derived 
forms with the analogous feature (ful- antonym). 
Some examples are colorful/less, fearful/less, harm- 
ful/less, and tasteful/less. 

The precision statistics for the individual lexical 
semantic features discussed above are presented in 
Table 1 and Table 2. Lexical semantic informa- 
tion was collected for 2535 words (bases and derived 
forms). One way to summarize these tables is to cal- 
culate a single precision number for all the features 
in a table, i.e., average the number of correct types 
for each affix, sum these averages, and then divide 
this sum by the total number of types. Using this 
statistic it can be said that if a random word is de- 
rived, its features have a 76 percent chance of being 
true and if it is a stem of a derived form, its features 
have a 82 percent chance of being true. 

Computing recall requires finding all true tokens 
of a cue. This is a labor intensive task. It was 
performed for the verbal prefix re- and the recall 
was found to be 85 percent. The majority of the 
missed re- verbs were due to the fact that the system 
only looked at verbs starting with RE and not other 
parts-of-speech, e.g., many nominalizations such as 
reaccommodation contain the re- morphological cue. 
However, increasing recall by looking at all open 
class categories would probably decrease precision. 
Another cause of reduced recall is that some stems 
were not in the Alvey lexicon or could not be prop- 
erly extracted by the morphological analyzer. For 
example, -Nize could not be stripped from hypoth- 
esize because Alvey failed to reconstruct hypothesis 
from hypothes. However, for the affixes discussed 
here, 89 percent of the bases were present in the 



Alvey lexicon. 



Feature 


Affix 


Types 


Precision 


TELIC 


re- 


164 


91% 


RSTATE-EQ-BASE- 


re- 


164 




65% 


RSTATE 










ENTAILS-BASE 


re- 


164 


65% 


PRESUPS-RSTATE 


re- 


164 


65% 


CHANGE-OF-STATE 


un- 


23 


100% 


NEG-OF-BASE-IS- 


un- 


23 




91% 


RSTATE 










CHANGE-OF-STATE 


de- 


35 


34% 


NEG-OF-BASE-IS- 


de- 


35 




20% 


RSTATE 










CHANGE-OF-STATE 


-Aize 


63 


78% 


RSTATE-EQ-BASE 


-Aize 


63 


75% 


CHANGE-OF-STATE 


-Nize 


86 


56% 


ACTIVITY 


-le 


71 


55% 


CHANGE-OF-STATE 


-en 


36 


100% 


RSTATE-EQ-BASE 


-en 


36 


97% 


CHANGE-OF-STATE 


-Aify 


17 


94% 


RSTATE-EQ-BASE 


-Aify 


17 


58% 


CHANGE-OF-STATE 


-my 


21 


67% 


CHANGE-OF-STATE 


-ate 


365 


48% 


PART-IN-E 


-ee 


22 


91% 


SENTIENT 


-ee 


22 


82% 


NON-VOLITIONAL 


-ee 


22 


68% 


PART-IN-E 


-er 


471 


85% 


PART-IN-E 


-ant 


21 


81% 


EVENT-AND- 


-age 


43 




58% 


RESULTANT 










REFERS-TO-E-OR- 


-ment 


166 




88% 


PROP-OR- RESULTANT 










INCORRECT-MANNER 


mis- 


21 


86% 


ABLE-TO-BE- 


-able 


148 




84% 


PERFORMED 










STATE- OF-HAVING- 


-ness 


307 




97% 


PROP-OF-BASE 










FUL- ANTONYM 


-less 


22 


77% 


LESS-ANTONYM 


-ful 


22 


77% 



Table 1: Derived words 
5 Evaluation 

Good surface cues are easy to identify, abundant, 
and correspond to the needed lexical semantic in- 
formation (Hearst (1992) identifies a similar set 
of desiderata). With respect to these desiderata, 
derivational morphology is both a good cue and a 
bad cue. 

Let us start with why it is a bad cue: there may 
be no derivational cues for the lexical semantics of 
a particular word. This is not the case for other 
surface cues, e.g., distributional cues exist for every 
word in a corpus. In addition, even if a derivational 
cue does exist, the reliability (on average approxi- 
mately 76 percent) of the lexical semantic informa- 
tion is too low for many NLP tasks. This unrelia- 
bility is due in part to the inherent exceptionality of 



Feature 


Affix 


Types 


Precision 


TELIC 


re- 


164 


91% 


CHANGE-OF-STATE 


Vun- 


23 


91% 


CHANGE-OF-STATE 


Vde- 


33 


36% 


IZE- DEPENDENT 


-Aize 


64 


80% 


IZE- DEPENDENT 


-en 


36 


72% 


IZE- DEPENDENT 


-Aify 


15 


40% 


ABSTRACT 


-fill 


76 


93% 



Table 2: Base words 



cueing that can be used in combination with others 
to provide lexical semantic information. 
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lexical generalization and thus can be improved only 
partially. 

However, derivational morphology is a good cue 
in the following ways. It provides exactly the type 
of lexical semantics needed for many NLP tasks: the 
affixes discussed in the previous section cued nomi- 
nal semantic class, verbal aspectual class, antonym 
relationships between words, sentience, etc. In ad- 
dition, working with the Brown corpus (1.1 million 
words) and 18 affixes provided such information for 
over 2500 words. Since corpora with over 40 million 
words are common and English has over 40 com- 
mon derivational affixes, one would expect to be able 
to increase this number by an order of magnitude. 
In addition, most English words are either derived 
themselves or serve as bases of at least one deriva- 
tional affixj^j Finally, for some NLP tasks, 76 per- 
cent reliability may be adequate. In addition, some 
affixes are much more reliable cues than others and 
thus if higher reliability is required then only the 
affixes with high precision might be used. 

The above discussion makes it clear that morpho- 
logical cueing provides only a partial solution to the 
problem of acquiring lexical semantic information. 
However, as mentioned in section 2 there are many 
types of surface cues which correspond to a vari- 
ety of lexical semantic information. A combination 
of cues should produce better precision where the 
same information is indicated by multiple cues. For 
example, the morphological cue re- indicates telic- 
ity and as mentioned above, the syntactic cue the 



progressive tense indicates non-stativity (Dorr and 
Lee, 1992). Since telicity is a type of non-stativity, 



the information is mutually supportive. In addition, 
using many different types of cues should provide a 
greater variety of information in general. Thus mor- 
phological cueing is best seen as one type of surface 



3 The following experiment supports this claim. Just 
over 400 open class words were picked randomly from 
the Brown corpus and the derived forms were marked 
by hand. Based on this data, a random open class word 
in the Brown corpus has a 17 percent chance of being 
derived, a 56 percent chance of being a stem of a derived 
form, and an 8 percent chance of being both. 
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